Fault-based testing without the need of oracles

There are two fundamental limitations in software testing, known as the reliable test set problem and the oracle problem. Fault-based testing is an attempt by Morell to alleviate the reliable test set problem. In this paper, we propose to enhance fault-based testing to alleviate the oracle problem as well. We present an integrated method that combines metamorphic testing with fault-based testing using real and symbolic inputs

Proportional sampling strategy : a compendium and some insights

There have been numerous studies on the effectiveness of partition and random testing. In particular, the proportional sampling (PS) strategy has been proved, under certain conditions, to be the only form of partition testing that outperforms random testing regardless of where the failure-causing inputs are. This paper provides an integrated synthesis and overview of our recent studies on the PS strategy and its related work. Through this synthesis, we offer a perspective that properly interprets the results obtained so far, and present some of the interesting issues involved and new insights obtained during the course of this research

Test case selection with and without replacement

Previous theoretical studies on the effectiveness of partition testing and random testing have assumed that test cases are selected with replacement. Although this assumption has been well known to be less realistic, it has still been used in previous theoretical work because it renders the analyses more tractable. This paper presents a theoretical investigation aimed at comparing the effectiveness when test cases are selected with and without replacement, and exploring the relationships between these two scenarios. We propose a new effectiveness metric for software testing, namely the expected number of distinct failures detected, to re-examine existing partition testing strategies

On the identification of categories and choices for specification-based test case generation

The category-partition method and the classification-tree method help construct test cases from specifications. In both methods, an early step is to identify a set of categories (or classifications) and choices (or classes). This is often performed in an ad hoc manner due to the absence of systematic techniques. In this paper, we report and discuss three empirical studies to investigate the common mistakes made by software testers in such an ad hoc approach. The empirical studies serve three purposes: (a) to make the knowledge of common mistakes known to other testers so that they can avoid repeating the same mistakes, (b) to facilitate researchers and practitioners develop systematic identification techniques, and (c) to provide a means of measuring the effectiveness of newly developed identification techniques. Based on the results of our studies, we also formulate a checklist to help testers detect such mistakes

An empirical evaluation and analysis of the fault-detection capability of MUMCUT for general Boolean expressions

Boolean expressions are extensively used in software specifications. It is important to generate a small-sized test set for Boolean expressions without sacrificing the fault-detection capability. MUMCUT is an efficient test case generation strategy for Boolean expressions in Irreducible Disjointed Normal Form (IDNF). In the real world, however, Boolean expressions written by a software designer or programmer are not normally in IDNF. In this paper, we apply MUMCUT to generate test cases for general Boolean expressions and develop a mutation-based empirical evaluation on the effectiveness of this application. The experimental data show that MUMCUT can still detect single seeded faults in up to 98.20% of general Boolean expressions. We also analyze patterns where test cases generated by MUMCUT cannot detect the seeded faults

An assessment of systems and software engineering scholars and institutions (2001–2005)

This paper presents the findings of a five-year study of the top scholars and institutions in the systems and software engineering field, as measured by the quantity of papers published in the journals of the field in 2001-2005. The top scholar is Magne Jørgensen of Simula Research Laboratory, Norway, and the top institution is Korea Advanced Institute of Science and Technology, Korea. This paper is part of an ongoing study, conducted annually, that identifies the top 15 scholars and institutions in the most recent five-year period

Toward a problem-driven approach to perspective-based reading

No abstract available